Advanced Intelligent Systems

Serial-section immunohistochemistry (IHC) is the largest available source of paired hematoxylin and eosin (HE) and IHC whole slide images, yet it remains underexploited for cell-level supervision: adjacent sections sample non-identical cells, and residual registration error prevents direct assignment of IHC labels to individual HE cells. We present CellDF (Cell Displacement Field), which turns registered serial-section data into pairs of HE cells and their IHC labels by solving cell matching at whole-slide scale and assessing its reliability without ground-truth correspondences. CellDF estimates a locally adaptive residual displacement field through iterated kernel regression over each HE cells K nearest IHC candidates; a sparse-kernel variant keeps it tractable at the cell counts of a whole slide, where pairwise matchers are not. The within-tile distribution of the estimated displacements yields two ground-truth-free statistics, the directional scatter{sigma}{theta} and the between-tile angular deviation |{Delta}{theta}|, that localize matching quality more finely than landmark-based target registration error and drive a two-stage outlier filter that withholds labels where matching is unreliable. On 54 same-section HyReCo pairs,{sigma}{theta} correlates only moderately with landmark error and flags localized restaining damage that global error misses; on 30 four-marker Acrobat serial-section cases, the same statistic flags which IHC marker, if any, lies physically close enough to HE to support cell-level transfer. As a proof of concept, IHC labels transferred through CellDF trained a cell classifier on HE embeddings that generalized to held-out cells within the sample (F1 0.85, AUROC 0.88), establishing serial-section IHC as a usable cell-level labeling resource. Graphical abstract O_FIG O_LINKSMALLFIG WIDTH=200 HEIGHT=78 SRC="FIGDIR/small/733058v1_ufig1.gif" ALT="Figure 1"> View larger version (42K): org.highwire.dtl.DTLVardef@a9b3dcorg.highwire.dtl.DTLVardef@15f652corg.highwire.dtl.DTLVardef@1eb3396org.highwire.dtl.DTLVardef@87dda2_HPS_FORMAT_FIGEXP M_FIG C_FIG

Three-dimensional (3D) imaging represents the development of next generation of fluorescence microscopy. However, routine axial down-sampling makes isotropic resolution unrealistic. Here, we propose DeepUI, a physical zero-shot framework designed to achieve isotropic 3D fluorescence images from a low axial sampling rate. DeepUI fully leverages the intrinsic characteristics of 3D images through physics-guided degradation, which incorporates spatial-frequency joint learning to generate a scaled optical transfer function, combined with noise degradation and an up-sampling branch. Typically requiring just 5 minutes for training and 0.5 minutes for high-throughput and fast prediction, we demonstrate the superior performance of DeepUI to get isotropic results, and the exclusivity to axial down-sampling conditions, even in more challenging conditions, including defocused background, noise, and resolution blur.

Optogenetics enables experiments in out-of-equilibrium conditions to clarify biological mechanisms and quantify biophysical parameters. However, modelling and control techniques to study mammalian cell biology under optogenetic perturbation remain underutilised. Here, we benchmark these methods within mammalian cells by steering nucleocytoplasmic transport via the optogenetic LEXY protein in outcome-driven microscopy. First, we employ system identification to obtain models that predict transport dynamics by minimising the prediction error. We quantify this prediction accuracy for one biophysical model and two black-box models. Second, we evaluate closed-loop control efficacy by steering transport along a predefined trajectory using model-free Proportional Integral (PI) control, model-based Linear Quadratic Regulation (LQR) and Model Predictive Control (MPC). Both the predictive models and the applied control techniques demonstrate robust performance against cell-to-cell variation. This biological variation is quantified by the parameter distributions obtained from model identification with single-cell trajectories. While we show that model-free techniques such as PI and gain-scheduled PI achieve steering without explict model knowledge, predictive architectures offer better performance under this cell-to-cell variation and time-varying setpoints. Moreover, black-box predictive accuracy suggests that this model-based control is possible, even when explicit mechanistic understanding is missing. Ultimately, we demonstrate that predictive modelling and optogenetics enable quantitative characterisation and precise manipulation of mammalian cells, while offering practical guidelines for the implementation of these techniques.

Due to its ability to establish a pool of undetectable and latently infected cells that can initiate viral production through random reactivation, a cure to human immunodeficiency virus (HIV) infections has remained elusive. Many approaches have been proposed, including the "shock and kill" method where latency reversing agents (LRAs) are administered to reactivate latently infected cells out of latency and remove them through immune targeting and clearance, and the "block and lock" method where latency promoting agents (LPAs) are administered to inhibit reactivation and potentially induce a "deep latency" state where infected cells can no longer reactivate. Previous large scale drug screen studies have demonstrated a correlation between a compound's capability to modulate the fluctuations (or "noise") in HIV gene expression and its potential to modulate HIV latency. However, measurements of gene expression noise are labor- and cost-intensive. To circumvent these drawbacks, we trained a variational autoencoder (VAE) on a previously published large scale time-lapse fluorescence microscopy dataset, and performed an in silico screening of ~175,000 compounds for HIV latency modulators. Out of the top 113 predicted modulators that were experimentally tested, 16 latency reversing agent (LRA) synergizers and 2 latency promoting agents (LPAs) were confirmed, yielding an overall experimental hit rate of 15.9%. Our work demonstrates that in silico drug screening modalities, guided by existing large-scale experimental datasets, can yield high experimental hit rates, reducing costs incurred from labor-intensive wet lab-focused methodologies.

While Single Molecule Localization Microscopy (SMLM) aims to generate precise coordinates of molecular targets in cells, the resulting point clouds are inherently blurred by additive noise sources across the experimental, imaging, and processing workflow. This blurring often limits SMLMs ability to accurately quantify complex assembled structures required to address biological issues, despite reported localization precision down to a couple of nanometers. Here, we present SMLMFlow, a machine learning framework for improving structural resolution in SMLM datasets that combines a graph neural network and a hierarchical transformer with flow matching. We show that SMLMFlow improves structural resolution and downstream quantification across different structures, including filaments and protein nano-clusters, and generalizes to new unseen photophysics models.

Optical fluorescence microscopy enables visualization of biological structures and dynamics. However, the intrinsic diffraction limit, especially axially, and depth-related scattering noise compromise the image resolution and fidelity. Computational 3D deconvolution is a promising approach for mitigating these issues, yet its execution is hindered by inaccurate and cumbersome theoretical modeling or experimental measurement of 3D point spread function (PSF), as well as ineffective 3D noise regularization. Furthermore, in the 3D super-resolution regime, there remains a lack of standardized tools for evaluating 3D super-resolution fidelity. Here, we present the 3D adaptive deconvolution and evaluation (3D-ADE) toolkit, which comprises 3D-Ada deconvolution with physics-oriented automatic 3D-PSF calibration, and 3D-SQUIRREL for 3D super-resolution quality assessment. It effectively resolves noise instability, eliminates the need for 3D-PSF calibration, and reliably assesses the fidelity of 3D resolution extension via deconvolution, physical, and deep-learning-based methods. Accessible via multiple software platforms, 3D-ADE enhances the versatility of 3D deconvolution and fills the gap in 3D super-resolution evaluation tools, and thereby advances volumetric fluorescence imaging applications.

Quantitative analysis of three-dimensional cellular architecture is fundamental to understanding tissue organization, disease progression, and drug response. Yet 3D cell segmentation remains a critical bottleneck due to diverse cell morphologies, low signal-to-noise ratios, and data scarcity. We introduce Penumbria, a general-purpose 3D cell segmentation framework that achieves state-of-the-art accuracy across morphologically distinct cell populations and imaging conditions in volumetric microscopy. Penumbria formulates segmentation as a regression problem on distances to cell boundaries, supporting instance reconstruction without shape priors and permitting end-to-end GPU inference. A U-Net-based architecture with xLSTM bottleneck blocks and patch embeddings enables multi-scale feature extraction, long-range modeling of spatial context, and convolutional feature-volume tokenization. The model is extended with two modules: a Global Zernike Phase Layer, which learns Zernike-parameterized phase corrections in the frequency domain to undo optical aberrations such as defocus and tilt, and a Scaled Geocaps Layer, which samples features at fixed grid locations across multiple spatial scales, routing evidence between them such that a detection is only confident where concordance holds across scales simultaneously. Across four diverse 3D datasets selected to probe the limits of existing methods, Penumbria outperforms Cellpose-SAM across all evaluation thresholds and surpasses StarDist-3D on most datasets while matching it on Parhyale hawaiensis. Trained entirely from scratch, Penumbria achieves up to a 38% improvement in mean average precision over the second-best method. Strong boundary accuracy further supports downstream analyses such as quantifying membrane dynamics or protein localization.

Foundation models (FMs) are central to digital pathology, encoding histology images into dense embeddings for facilitating diagnostic classification, molecular alteration prediction, and clinical outcome modeling. However, the opacity of these embeddings renders FM-based systems "black boxes," limiting their trustworthiness for clinical translation and utility for scientific discovery. Here, we introduce PICASSO (Pathology Image Concept Atlas built via SparSe dictiOnary learning), a framework that makes pathology FMs interpretable and controllable. PICASSO decomposes FM embeddings into human-interpretable visual concepts using a sparse autoencoder. It is trained on more than 120 million tissue patches across 32 cancer types, producing the first pan-cancer atlas of histomorphological concepts. We demonstrate that PICASSO enables diverse downstream applications of FM embeddings by exposing interpretable structure within learned representations and supporting concept-level intervention. It enables auditing of clinical model behavior by revealing the morphological features driving predictions. Beyond transparency and validation, PICASSO enables the discovery of new biological insights; for example, it identified hobnailing epithelial morphology as a previously unrecognized biomarker of EGFR mutations in lung adenocarcinoma. By linking PICASSO-derived concepts with spatial transcriptomics, we uncover associations between morphological patterns and gene expression programs. Furthermore, PICASSO allows suppression of concepts associated with technical artifacts, thereby reducing model reliance on spurious signals. Finally, PICASSO enables controlled manipulation of learned concepts to generate counterfactual embeddings for exploratory therapeutic analysis, such as modulating tumour-infiltrating lymphocyte density to assess impacts on predict survival outcomes. Together, PICASSO provides a principled framework for transforming pathology FMs into platforms for mechanistic insight and discovery.

Deep learning models based on Vision Transformers (ViTs) have shown strong performance in retinal fundus imaging, but their interpretability remains poorly understood. In particular, attention-based attribution methods are widely used to explain ViT predictions, despite limited evaluation of their faithfulness and biological relevance in medical imaging. Here, we systematically benchmark four attention-based interpretability methods for RETFound, a retinal ViT-based foundation model, that we previously fine-tuned to predict 17 retinal vascular phenotypes from UK Biobank fundus images1. We compare raw attention, attention rollout, gradient-weighted attention rollout, and Chefers hybrid relevance-based method using both qualitative visualisation and quantitative evaluation frameworks. To assess attribution faithfulness, we perform perturbation-based deletion and insertion experiments, quantifying changes in model predictions as highly attended image regions are progressively removed or restored. To evaluate biological specificity, we run structure-aware analyses combining attribution maps with vessel segmentation and artery-vein labels through the Relative ratio of Attention Intensity (RAI) metric. Across models, attribution maps differed substantially depending on the selected interpretability method, highlighting the need for rigorous quantitative evaluation. Among the evaluated approaches, gradient-weighted attention rollout consistently achieved the strongest perturbation performance and produced attribution maps most closely aligned with the anatomical definition of the predicted retinal traits. Furthermore, vessel-type specific models systematically concentrate attention on the corresponding vascular structures despite being trained using only a single scalar value per image as supervision. These findings demonstrate that attention-based attribution methods capture biologically meaningful vascular representations, while also revealing method-dependent variability in attribution behaviour. This work provides a quantitative framework for evaluating interpretability methods in medical imaging with annotated segmentation and contributes toward more transparent and biologically grounded medical AI systems.

Light-sheet fluorescence microscopy (LSFM) has revolutionized biological imaging by enabling high spatial and temporal resolution with minimal photodamage. However, conventional LSFM techniques often suffer from striping artifacts in the resulting images due to light scattering and absorption within samples, leading to uneven illumination that negatively impacts the accuracy of subsequent image analyses. To address this limitation, we introduce dodecagon light-sheet fluorescence microscopy (dodecaLSFM), a novel approach that maximizes angular diversity to achieve homogeneous illumination and suppress striping artifacts. dodecaLSFM employs diffraction optics and cylindrical lenses to generate twelve light sheets, providing 360 degree omnidirectional illumination that significantly enhances illumination uniformity compared to traditional mSPIM, mDSLM, and ultramicroscopy systems, which use only one or two illumination planes. We demonstrate the effectiveness of dodecaLSFM by achieving high-resolution imaging of whole mouse brain vasculature following tissue clearing, allowing precise morphometric analysis of vascular networks without striping artifacts. Furthermore, we show that combining dodecaLSFM with expansion microscopy (ExM) enables whole-organ 3D imaging at cellular resolution. This novel approach provides an advanced, scalable solution for large-volume imaging, facilitating detailed structural and functional studies across diverse biological applications.

Label-free tracking of adherent cell migration could enable important insights into biological processes such as tissue repair, inflammatory response, or cancer progression. Nevertheless, visualizing unlabeled animal cells using optical microscopy remains challenging due to low contrast as well as frequent changes in cell shape and number. A promising alternative uses electrical capacitance measurements, which are sensitive to cell adhesion to electrode surfaces. However, prior examples often utilized electrodes with areas larger than single cells, resulting in averaged readouts over multiple cells. Here, we demonstrate label-free, live-cell tracking using a capacitance sensor array with more than 1 million pixels on a 10 micron pitch across an area larger than 1 square centimeter. We show that single cell morphology can be clearly segmented, and then used to reconstruct migration and proliferation dynamics using optical flow. We further track the spreading of multicellular spheroids, revealing fast-moving peripheral regions led by a collective leader cell "front." Finally, we demonstrate label-free imaging of millimeter-scale honeycomb-shaped tissues without the multi-image stitching often required for conventional microscopy. We utilize mutual capacitance measurements with electrically-programmable electrode spacing to reconstruct topographical features of these engineered tissues. Overall, CMOS capacitance imaging arrays enables label-free imaging spanning from single cells to large tissues, in a portable and scalable format for settings where optical microscopy may be difficult to access.

Designing transcutaneous skeletal muscle oxygenation (SmO2) sensors requires jointly optimizing source--detector geometry and wavelength selection while guaranteeing performance across populations that vary in subcutaneous fat thickness and skin pigmentation. We present a multi-fidelity Bayesian optimization (MFBO) framework that couples Monte Carlo light-transport simulations at two photon-count fidelities to a distributionally robust design objective. An autoregressive Gaussian-process surrogate learns the correlation between inexpensive low-photon-count and accurate high-photon-count simulations, and a cost-aware acquisition function decides both where and at what fidelity to sample. Robustness across the population is enforced with Conditional Value-at-Risk (CVaR) and entropic-risk (ERM) objectives that target worst-case subjects rather than the population average. On a five-layer forearm tissue model with anthropometric variability we find (i) a fidelity regime that is favorable for MFBO where the low-fidelity surrogate is rank-informative (Spearman {rho} = 0.84) but biased, at 100x lower cost; (ii) MFBO attains 23% higher robust sensitivity than a strong high-fidelity single-fidelity baseline at equal budget (p = 0.035), and avoids the optimistic bias that causes low-fidelity-only optimization to collapse when its designs are validated at high fidelity; (iii) CVaR/ERM objectives improve worst-case tail performance by {approx}23% relative to a mean objective without sacrificing average sensitivity; and (iv) discovered designs improve robust tail sensitivity by roughly 3--6x over commercial and heuristic optode layouts, with the largest gains in the high-fat and high-melanin subpopulations. The methodology bridges stochastic light-transport physics with sample-efficient machine-learning optimization and generalizes to cerebral oximetry, photodynamic therapy planning, and wearable physiological monitors.

Lipid Nanoparticles (LNPs) are widely used as delivery systems for nucleic acid therapeutics, where transfection efficiency is determined by both the identities of constituent lipid components and their composition ratios. While prior studies have focused on learning molecular representations for individual components, modeling how multiple components and their ratios jointly influence LNP performance remains underexplored. In this work, we propose STRATA, a framework that models molecule interaction between LNP components, which is known to contribute to LNP transfection efficiency. Our approach is built on two complementary views: (1) a ratio-centric view that captures interaction patterns induced by composition ratios through a transformer with a Ratio-induced Positional Embedding, and (2) a molecule-centric view that incorporates interaction-induced effects into structure-based molecule embeddings. By jointly training and aligning these views, our model integrates molecular structure and composition ratio within a unified framework that captures interaction-driven effects. Experiments demonstrate that our method improves prediction accuracy and generalization to unseen molecules and ratios, highlighting the effectiveness of our approach. Implementation code will be available after acceptance.

Single-molecule (SM) imaging (SMI)-based approaches have the powerful ability to capture receptor interactions, which are necessary for cell signaling, in their native live-cell environment. Yet, due to substoichiometric labeling, SMI generally provides only partial information on these interactions. We developed Deep-FISIK, which utilizes graph neural networks and multi-head attention for message-passing, to predict from SMI data the kinetics of homotypic interactions of the full receptor system. The input to Deep-FISIK are the SM detections in SMI experiments, without the need for explicit tracking. Thus, Deep-FISIK is compatible with labeling a higher fraction of receptors in the SMI experiments, increasing the prediction accuracy of the interaction kinetics parameters. The performance of Deep-FISIK is robust in the presence of a variety of deviations from the training data, indicating the applicability of Deep-FISIK to many receptor systems and SMI experiments.

Microscopic images of cells and tissues are central to disease diagnosis. In computational pathology, multiple instance learning (MIL) has emerged as a key paradigm for analyzing numerous images within a single patient sample. While the representative distribution of cells in a sample is important for diagnosis, existing MIL frameworks largely overlook it. We introduce TopoMIL, a framework that extracts the representative topological structure of the sample and integrates it into the MIL classifier. Three topological representations are assessed, each with distinct advantages and computational costs. We evaluate TopoMIL on four histopathology and cytomorphology datasets, each presenting unique challenges. Integrating the samples topological information into MIL enhances classification across average, max, attention-based, and transformer pooling, yielding AUCROC gains of 3.3%, 4.2%, 5.9%, and 0.5%, respectively, with moderate computational cost. Our work underscores the potential of TopoMIL as a scalable extension to existing morphology-based models in computational pathology.

Precise, non-invasive manipulation of individual living cells remains a central challenge in biomedical science, with far-reaching implications for single-cell analysis, tissue engineering, and the study of cell-cell interactions. Here, we report the first demonstration of single-cell control using bulk acoustic standing-wave acoustofluidics with closed-loop feedback. We introduce VeLO (Vector-based Local Optimization), a model-free, reinforcement learning-inspired algorithm that enables programmable two-dimensional manipulation of individual cells using a single piezoelectric transducer. Without prior calibration or physical modeling, VeLO learns system dynamics online from acoustically induced cell displacements and automatically adapts to nonlinear, time-varying conditions. We achieve robust control across multiple cell types (DU-145, Jurkat, K-562) and independent manipulation of multiple cells, including controlled cell-cell contact. By combining simplicity of hardware with autonomous, adaptive control, this approach establishes multimodal acoustofluidics as a versatile tool for label-free, high-precision single-cell manipulation.

Advances in fluorescence microscopy have dramatically expanded the range of biological questions that can be addressed, enabling quantitative observations of molecular interactions and cellular dynamics with unprecedented spatial and temporal resolution. However, the growing complexity of imaging data has outpaced our ability to analyze them. Despite numerous computational methods exist, they often rely on specialized software environments, heterogeneous data formats, and technical expertise, limiting adoption and widening the gap between data acquisition and quantitative biological interpretation. Here we introduce BioBrain, a multi-agent framework that translates natural-language analytical goals into executable and reproducible microscopy analysis pipelines. Instead of generating analysis code, BioBrain assembles validated analytical methods and can expands its analytical capabilities by integrating existing laboratory scripts into a unified conversational framework. Every selected method and inferred parameter is transparently reported, ensuring traceable and reproducible analyses. On two-channel total internal reflection fluorescence and three-dimensional lattice light-sheet benchmarks, BioBrain exactly reproduces expert-derived results when parameters are specified and degrades predictably and traceably when they are not, while frontier language models generated large, model-dependent quantitative errors despite completing without warning. BioBrain offers a practical path for closing the widening gap between data acquisition and biological discovery, enabling experimental scientists to communicate with computational analysis in the language of biology rather than the language of software.

Label-free optical redox imaging utilizes endogenous NAD(P)H and FAD autofluorescence to evaluate metabolism in living specimens. The conventional optical redox ratio collapses these two channels into a single value; however, it does not indicate whether a pixel has sufficient photon support or the cellular context necessary for quantitative aggregation. To address this limitation, we introduce FPhaS, a fixed-calibration phase- autofluorescence framework that integrates quantitative phase imaging (QPI) with simultaneous label-free autofluorescence multi-harmonic microscopy (SLAM), using fluorescence lifetime imaging (FLIM) solely for validation. Because QPI and SLAM are acquired with the same objective, a unified non-biological calibration aligns phase-derived structural data with the autofluorescence frame, yielding a residual error of 0.39 pixels. This calibration is maintained across all biological specimens. This shared geometric reference enables local evaluation of structural and metabolic information, rather than comparing approximately aligned images. FPhaS decomposes the data into cell presence, ratio credibility, and confidence-supported pooling. We validated FPhaS on A549 cells under high and low-photon conditions; the framework is designed to generalize to other cell and tissue types. Confidence-weighted intensity redox estimates were compared with lifetime-derived measurements within mask-locked cellular regions. Concordance improved exclusively when both the denominator photon support and an independent structural criterion were satisfied. The same reference layer generated cell-level descriptors of metabolic content, metabolic-structural organization, and measurement reliability, while also constraining the CombinedWLS reconstruction under diminished fluorescence acquisition. FPhaS redefines label-free metabolic imaging from producing comprehensive ratio maps to identifying regions where optical evidence substantiates quantitative inference.

Minimally invasive surgery is a powerful technique that enables operations deep within the body while minimizing patient trauma and recovery time. Optical endoscopes are key to providing intraoperative vision but still face challenges due to the loss of essential senses, including depth perception and tactile feedback for tissue evaluation. Thus, it is critical to develop endoscopic imaging technologies that can augment operators with critical information. In this work, we explore a prototype multimodal 3D imaging endoscope that integrates volumetric light-field imaging with laser-speckle contrast imaging to simultaneously capture 3D structure and blood-flow information in a clinically relevant form factor.

Deep generative models have become a leading approach for designing therapeutic molecules, yet efficiently exploring vast biomolecular sequence spaces remains difficult, particularly for targets with limited training data. The prior distribution that seeds a generative model shapes which regions of sequence space it explores, and recent work suggests that non-classical distributions sampled from quantum processors can serve as a structured alternative to the factorised Gaussian priors used by default. Whether such priors help on complex biological design tasks has been largely untested. Here we present the first end-to-end hybrid quantum-classical pipeline for de novo design of MHC class I-binding peptides, coupling a generative adversarial network (GAN) to latent vectors sampled from a real photonic quantum processor. Tested in silico across 131 HLA alleles, quantum-derived priors increased the yield of predicted strong binders, with the largest relative gains for understudied alleles where classical baselines perform worst. We selected three understudied alleles for further evaluation, finding that large gains coincided with broader sequence exploration at non-anchor positions while anchor specificity was preserved. On these three alleles, we validated the designs in vitro using peptide-MHC stability ELISAs, confirming that quantum-designed peptides are potent stabilisers of peptide-MHC class I complexes. These results establish structured, hardware-realisable non-classical priors as a useful inductive bias for generative peptide design, with direct relevance to personalised immunotherapies and vaccines.